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Method of generating synthetic key frame and video browsing system using the same 



(57) There are provided a method of generating a 
synthetic key frame, capable of displaying lots of Infor- 
mation on limited device, and a video browsing system 
using the synthetic key frame. The method of generating 
a synthetic key frame includes tne steps of receiving a 



video stream from a first source and dividing it into 
meaningful sections, selecting key frame(s) or key re- 
gion^) representative of a divided section, and combin- 
ing the selected key frame(s) or key region(s) to gener- 
ate one synthetic key frame. 
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Description 

BACKGROUrJD OF THE INVENTION 

1 Field of the Invention 

[OOOl] The present Invention relates to a content- 
based multimedia searching system and. more particu- 
larly, to a synthetic keyframe generating method capa- 
ble of displaying lots of Information on a screen with a 
faed size and a video browsing system using thereof. 

2 Description of the Related An 

[0002] With the development of Image/video process- 
ing technologies in recent years, users can search/filter 
and browse a desired part of a desired video contents 
(or moving picture, for example, movie, drama, docu- 
mentary program, etc.) at a desired time. 
[OOQS] A basic technique for non-linear video brows- 
ing or searching includes shot segmentation and snot 
clustering. These techniques are used for analyzing and 
searching or browsing multimedia contents. 
[0004] In the image/video processing technologies, a 
shot Is a sequence of video frames obtained by one 
camera without interruption, which Is a basic unit for 
constructing and analyzing a video. A scene is a con- 
stituent element meaningful in the video, that is, signif- 
icant element in the development of story One scene 
Includes a number c-f shots. 

[0005] Meanwhile, a video indexing system structur- 
ally analyses video contents and detects shots and 
scenes using a shot segmentation engine and a shot 

' clustering engine. The video indexing system also ex- 
tracts keyframes orkey regions capable of representing 
a segment based onthe detected shots and scenes , and 
provides a too! for summarizing the video stream or di- 
rectly moving to a desired position in the video stream. 
[0006] FIG- 1 shows structural information of a gen- 
eral video stream. Referring to FIG. 1 , a video stream is 
consist of a series of scene that is a logical story unit 
regardless of video genre, each scene is composed of 
a plurality of sub-scenes or shots , and each sh ot is com- 
posed of sequence of frames. 
[0007] Most video indexing systems extract shots 

• from the video stream and detect scenes based on the 
extracted shots, thereby indexing structural information 
of the video stream That is, the video indexing system 
extracts a Key frame (a video frame extracted from the 
video stream In order to represent a unit segment well) 
or key region, and index data for summarizing/search- 
ing/orowsing video contents 
[000B] FIG. 2 shows the relationship between an an- 
ch or frame and a key region In a news content according 
to a prior art. A news icon In the anchor frame F-an con- 
sisting of a image or characters for summarizing a news 
segment represents contents of anchor shot or corre- 
sponding news article. When it is selected as a key re- 
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gion Reg-K, it Is a component representing the corre- 
sponding segment That is. the key region Reg-k means 
a region which is capable of concisely representing con- 
tents of a particular segment such as text, human face. 

5 news icon, 

[0009] FIG. 3 shows a conventional non-linear video 
browsing interface which includes a video reproduction 
view V-VD, a key frame view V-Fk displaying one-di- 
mensionally key frames representing each shot or each 

10 scene, and a tree-shaped table of content (TOC) view 
V-TOC for directly providing structural information of .a 
video stream to users. Here, each of nodes (ND) of the 
tree-shaped TOC is a shot and scene representing con- 
tents included lowertrees and it means a keyframe. Ac- 

15 cordlngly. the interface allows a user to be able to easily 
move to a desired part of a video or to select and browse 
a desired part in the video stream without watching the 
whole content. 

[0010] However, the above-described conventional 
20 video browsing system that represents partial sequenc- 
es bythe key frames or key regions to index/summarize/ 
browse the video has the following problems. 

1 ) The conventional system cannot display relative- 
25 fy lots of information on a screen having a fixed size 
The conventional key frames and key regions using 
in the non-linear video browsing system and in the 
universal multimedia access applications (UMA) 
are used as means for transmitting the summarized 

so content of a video stream to users through Images. 
However, the users cannot grasp th© whole con- 
tents of th© video stream through the key frames or 
key regions in small numbers, displayed on the 
screen having a fixed size. One shot Includes video 

35 frames displayed forseveral to tens seconds and a 
scene is configured of shots although it depends on 
the genres or characteristics of programs included 
in the video. Incase of a shot that is long or severely 
variable, thus, one key frame is not appropriate for 

40 representing this shot. Accordingly, multiple key 
frames should be set for one shot or scene. 

Furthermore, In case where relatively large 
numbers of keyframes are provided to a TV or po- 
table terminal that cannot display a lot of key frames 

45 on a screen With a toced Size at a time in order to 
represent the whole contents of shot and/or scene, 
the user should operate his/her input device many 
times because he/she has to browse the lots of key 
frames. The number of the key frames may be re- 

so duced to solve this problem. In this case, however, 
key frames ih small numbers cannot represent the 
content of the video stream, as described above. 
Accordingly, there is required an efficient user inter- 
face capable of displaying lots of Information on a 

55 screen with a fixed size 

2) It is difficult that the content of a 3cene Including 
shots or sub-scenes is selected as one Keyframe. 
That is, generally, it is difficult to select a key frame 
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concisely representing contents of a scene. 

[0011] Accordingly, there is needed a new method of 
summarizing a video stream having a hierarchical struc- 
ture to allow keyframes of upper structures to satisfac- 
torily reflect contents included in tower structures. 

SUMMARY OF THE INVENTION 

[00121 it would therefore be desirable to provide a 
method of generating a synthetic key frame, which is 
capable of representing tots of information on a screen 
with a fixed size. 

[0013] It would also be desirable to provide a method 
of describing a synthetic key frame logical ly or physically 
formed by combining key frames or key regions. 
[0014] It would also be desirable to provide a method 
of summarizing a video hierarchically using a synthetic 
Key frame. 

[0015] It would also be desirable to provide a Video 
browsing Interface using a synthetic key frame, 
[001 6] It would also b© desirable to provide a non-lin- 
ear video browsing method using a synthetic keyframe, 
[0017] It would also be desirable to provide a data 
managing method using a synthetic key frame. 
[0018] The invention provides a method of generating 
asynthetic key frame, comprising the steps of: receiving 
a video stream from a first source and dividing it Into 
meaningful sections; selecting key fram©(s) or key ra- 
gion(s) representative of a divided section; and combin- 
ing the selected key frame(s) or key region(s). to gen- 
erate on e synthetic key frame 
[0019] The invention also provides a metnod of de- 
scribing synthetic key frame data, comprising the steps 
of: dividing a video stream into meaningf u I sections, and 
synthesizing a key frame or key region representing the 
content of each section Into one image to generate a 
synthetic key frame; and describing a list of key frame/ 
key region included in constituent elements of the syn- 
thetic key fr ame. 

[0020] The invention also provides a method of de- 
scribing synthetic key frame data, comprising the steps 
Of dividing a video stream into meaningful sections, and 
synthesizing a key frame or key region representing the 
content of each section into one image to generate a 
synthetic key frame; and generating a combination of 
key frames or key regions, or key frame and key region 
included In constituent elements of the synthetic key 
frame, and physically storing the combination to de- 
scribe the synthetic key frame 
[0021] The invention also provides a hierarchical vid- 
eo summarizing method using a synthetic key frame, 
comprising the steps of: dividing a video stream into 
meaningful sections, .and synthesizing a key frame or 
key region representing the content of each section Into 
one image to generate a synthetic key frame, and as- 
signing the synthetic key frame to a key image locator, 
a hierarchical summary listfor describing lowersumma- 



ry structures, and structural information of the video 
stream 

[0022] The invention also provides a method for pro- 
viding a video browsing interface, comprising the steps 

s of dividing a video stream into meaningful sections, and 
synlhesizing a key frame or key region representing the 
content of each section into one image to generate a 
synthetic key frame; and providing a user interface to a 
predetermined display to browse a video related with the 

10 generated synthetic key frames . 

[0023] The invention also provides a non-linear video 
browsing method, comprising the steps of; dividing a 
video stream into meaningful sections, and synthesizing 
a key frame or key region representing the content of 

is each section into one image, to generate a synth etic key 
frame; providing a user interface to a predetermined dis- 
play to browse a video related with the generated syn- 
thetic key frames; selecting the synthetic key frame ac- 
cording to an input by a user; and reproducing a seg- 

so ment represented by the selected synthetic key frame. 
The invention also provides apparatus, computer pro- 
gram products, interfaces and data structures corre- 
sponding to the above methods. 

& BRIEF DESCRIPTION OF THE DRAWINGS 

[0024] Amorecomplete appreciation of the invention, 
and many of the attendant advantages thereof, will be 
readily apparent as the same becomes better under- 
go stood by reference to the following detailed description 
when considered in conjunction with the accompanying 
drawing, in which like reference symbols indicate the 
same orthe similar components, wherein; 

35 FIG, 1 shows structural information of a general vid- 
eo stream; 

FIG 2 shows the relationship between an anchor 
frame and a news icon in a prior art; 
FIG. 3 shows a conventional non-linear video 
40 browsing interface; 

FIGS. 4A and 4B are diagrams for explaining the 
concept of a synthetic key frame according to the 
present invention; 

FIG 5A shows the description structure of a seg- 
45 ment locator according to the present invention, 

FIG SB shows the description structure of an image 
locator according to the present invention; 
FIG 6 shows the description structure of a keyframe 
locator according to the present invention; 
5Q FIG 7 shows the description structure of a key re- 
gion locator according to the present invention; 
FIG 8 shows the description structure of synthetic 
key frame information according to the present In- 
vention; 

55 FIG. 9 shows the description structure of a layout 
with respect to the arrangement of constituent ele- 
ments of a synthetic key frame according to the 
present invention, 
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FIG. loshows thestfucture of a news video accord- 
ing to the present invention; 
RG 11 shows a synthetic key frame of news head- 
lines according to the present invention, 
FIGS 12A and 128 show synthetic key frames 
detailed news sections according to the present in- 
vention; 

FIGS. 13Aand 13B show synthetic key frames gen- 
erated from a soccer game video according to the 
present invention; 

FIG 1 4 shows structural information of a video and 
hierarchical synthetic key frames according to the 
present invention; 

F!G. 15 shows the description structure of a hierar- 
chical image summary element for hierarchical vid- 
eo stream summary according to the present inven- 
tion; 

FIG, 16 shows a video browsing interface using a 
synthetic key frame according to the present inven- 
tion; 

FIG. 1 7 shows an example of application of the syn- 
thetic key frame according to the present invention 
to UMA; and 

FIG. 18 is an example of a flow diagram showing a 
method of communicating information using the 
synthetic key frame according to the present inven- 
tion, applied to UMA. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENT 

[0025] Reference will nowbe made in detail to the pre- 
ferred embodiments of the present Invention, examples 
of which are illustrated in the accompanying drawings, 
[0026] FIGS. 4A and 48 are diagrams for explaining 
the concept of a synthetic key frame according to the 
present invention. Refeningto FIG 4A, the synthetic key 
frame according to the invention is generated by com- 
bining key frames or key regions Reg-k from frames Fl. 
Fm, Fn which are extracted at predetermined points of 
time tl, tm, tn within one segment Sgti when a video 
stream is divided Into predetermined numbers of seg- 
ments Sgti , Sgi2 ( ..~, Sgti. Sgti+1 . Referring to FIG4B. 
the synthetic keyframe of the invention is generated by 
combining key framesorkey regions Reg-k from frames 
Fo, Fp, Fq, Fr extracted at predetermined points of time 
to,'tp. tq, tr within one segment Sgtj+1 and external 
frames Fext supplied from an external source when a 
video stream is divided into predetermined numbers, of 
segments Sgti , Sgi2, Sgtj, Sgtj+1 . , 
[0027] The synthetic key frame of the invention, dif- 
ferent from the key frame rn the prior art, is not a frame 
which has been physically generated in the video 
stream because It is created by combining regions hav- 
ing meaningful Information or key frames In orderto rep- 
resent a specific segment in the video stream 
[0028] FIGS. 5A and 5B respectively show descrip- 
tion structures of a segment locator and an image loca- 



tor according to the present invention. Referring to FIG 
6A, the segment locator as a means for designating a 
segment in a video stream, inlcudes segment lD r Media 
URL or actual segment data for desiganting the aodk>~ 
£ visual segment and segment time information such as 
segment starting/ending time or length, description in- 
formation for annotation for the segment, and a related 
segment list. 

[0029] Here, the related segment list is used for rep- 
10 resenting description of abstract/detail, cause/result re- 
lation among segments, and components of the list in- 
clude variables such as the segment locator or an iden- 
tifier for referring to the segment locator. 
mo30] Referring to FIG. 5B, the image locator as a 
13 means for designating an image includes inherent ID, 
image URL, or Image data for designating the Image. 
The image locator can havea structure which is capable 
of describing information such as an image related seg- 
ment list and annotation. 
2Q po31] FIG 6 shows the description structure of a key 
frame locator according to the present invention. As 
Shown in FIG. 6, th e keyframe locator includes an image 
locator, additionally, a representative segment locator 
for indicating which segment is represented by eorre- 
25 spoiling key frame, and fidelity values for indicating 
how faithfully corresponding segment is represented. 
[0032J FIG 7 shows the description structure of a key 
region locator according to the present invention, which 
is a logical or physical key region description structure. 
so p033] The logical key region description structure in- 
cludes an ID, an image locator, and region area infor- 
mation corresponding to a key region of an image des- 
ignated by the image locator. It additionally includes a 
representative segment locatorfor indicating which scg- 
35 ment is represented by the corresponding key region, 
fidelity valuesfor indicating howfafthfulry the key region 
represents corresponding segment, description infor- 
mation for other annotations and a related segment list 
for designating segment related with the key region. 
40 This logical key region description structure describes 
the key region using metadata. 
[0034] The physical key region description structure 
includes an inherent ID, region data, a representative 
segment locator for indicating which segment is repre- 
ss sented by corresponding key region if required, fidelity, 
description and a related segment list. For the video 
browsing interface using the synthetic key frame ac- 
cording to the present invention, the synthetic key frame 
must have been physically generated or be logically de- 
so scribed in a content-based data region with respect to 
a video stream. 

[0035] FIG 8 shows the description structure of syn- 
thetic keyframe information according to the present in- 
vention, which has a logical description structure and a 
55 physical description structure, 

[0036] As shown in FIG. 8, the logical synthetic key 
frame description structure includes variables such as 
an ID, a representative segment locator for designating 
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a segment represented by the synthetic key frame, a 
key frame list and a Key region list that are constituent 
elements of the synthetic icey frame, fidelity f or indicat- 
ing how faithfully the synthetic key frame represents the 
segment, and layout information for indicating the ar- 
rangement state of constituent elements of the synthetic 
key frame 

[00371 The physical synthetic key frame description 
structure includes variables such as an ID, an Image lo- 
cator for designating the actual synthetic key frame, a 
representative segment locator for designating a seg- 
ment represented by the synthetic key frame, fidelity for 
indicating how faithfully the synthetic key frame repre- 
sents the segment, a key region list related with the syn- 
thetic key frame, and layout information for indicating 
the arrangement state of constituent elements of the 
synthetic key frame. 

[0038] Here, keyframe elements constructing the key 
frame List include a keyframe locator for designating a 
corresponding key frame and fidelity for indicating how 
important meaningful information the corresponding key 
frame represents In the syntnetic key frame structure, 
as shown in FIG- 8. Furthermore, key region elements 
constructing the key region list include a key region lo- 
cator for designating a corresponding key region and fi- 
delity information for indicating how important meaning- 
ful information the corresponding key region represents 
In the synthetic key frame structure. The fidelity can be 
extracted automatically or manually. The fidelity auto- 
matical ly extracted is obtained with regard to informa- 
tion like duration of the key region, the size of an object, 
audio, etc. and a matching level of these information 
Items 

[0039] FIG 9 shows the description structure of layout 
information with respect to the arrangement of constit- 
uent elements of the synthetic key frame according to 
the present Invention. This description structure Is rep- 
resented by a markup language such as HTML and 
XML. Because the constituent elements of the synthetic 
keyframe may be arranged, being overlapped, the lay- 
out description structure includes layer information 
aboutthe first layer (iayer=0), the second layer (layer=1) 
and so on, and information about a location where the 
key frame or key region contained in each layer is dis- 
played or to be displayed on a screen. 
[0040] There will be explain ed an example of applica- 
tion of the synthetic key frame structure end synthetic 
keyframe generating method according to the invention 
to a broadcasting program. 

A) Synthetic key frame generated from a news video 

[0041] FIG. 1 0 shows the structure of the news video 
according to the present invention. The news video is 
generally configured of a headline news section NS-HL, 
a detailed news section NS-DT, a summary news sec- 
tion and a weather/sports section A commercial adver- 
tisement section may be added thereto Each of these 



sections further includes sub-sections. The section cor- 
responds to a scene in the video stream structure For 
example, the headline news section NS-HL may be di- 
vided into headline items HL-it and the detailed news 

5 section NS-DT may be classified into news items DT-it 
Here, the items canbeformed of keyframes. Each news 
item DT-it is basically divided into an anchor scene Scn- 
an and an episode scene Scn-ep 
[0042] FIG. 11 shows an example of a process of gen- 

10 erating the synthetic key frame of headline news section 
NS-HL according to the present invention. 
[0043] TVie headline news section NS-HL is construct- 
ed of f ive headline items HL-it These headline Items are 
configured of twenty-three shots and the running time is 

is 59 seconds, approximately. The five headline items are 
summarised using key frames F1, F2> F3, F4 and F5 
extracted at points of time H . t2, t3, t4 and t5, respec> 
tlvely Accordingly, one synthetic keyframe Fsk accord- 
ing to the present invention is created in a manner that 

20 key regions Regt, Reg2, Reg3, Reg4 and Regs, con- 
figured of texts, are extracted from the key frames F1 , 
F2, F3> F4 and F5 to be combined. The synthetic key 
frame can display the whole contents of the headline 
news section NS-HL on a screen with a fixed size at a 

25 time. 

[0044] On the contrary, the conventional video index- 
ing system should select several key frames represent- 
ing the headline news section, for example, because it 
assigns at least one key frame to an individual shot or 

00 scene. Furthermore, it cannot display an entire contents 
of headline section on a screen at a time. 
[Q045] Figs 12A and 12B show synthetic key frames 
of detailed news sections according to the present In- 
vention. FIG. 12A illustrates a synthetic key frame Fsk 

ss formed from one news Item NS-it that is constructed of 
twenty-one shots and fifty-seven seconds long, and 
FIG. 12B Illustrates asynthetic keyframe Fsk extracted 
from one news Item NS-it that is constructed of twenty- 
one ©hots and one-hundred-seven seconds long. That 

40 is, the synthetic key frames corresponding to news 
items of a news program can be differently formed. 
Where the synthetic key frames are arranged or allocat- 
ed to corresponding nodes in the TOG interface, the 
contents of lower structures of theTQC interface can oe 

4S displayed at a time. On the contrary, the conventional 
video indexing system should extract lots of key frames 
for a single news item and ft cannot display these key 
frames on a screen at the same time 

so B) Synthetic key frame generated from a sports video 

[0046] Other than news, it is necessary to summarize 
streams base on segment-based summary in sports 
news. For example, soccer video is configured of great 
S3 numbers of videoframes so thatthe running time is long 
To summarize the soccer video accordingly, one shot 
should be represented by lots of key frames and one 
key frame is difficult to represent a scene constructed 
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of shots 

[00471 F, G S - 1 3A and 1 3B show synthetic key frame© 
generated from the soccer game video according to the 
present invention. 

[00481 F lG - 1 3A illustrates a synthetic key frame Fsk 
generated from one scene constructed of nine shots 
whose running time is sixty-five seconds, and FIG- 1 3B 
illustrates a synthetic key frame Fsk generated from one 
scene constructed of nine shots whose running time is 
fifty-three seconds, 

[0049) Though the shots Included in one scene have 
different contents, the synthetic key frame Fsk accord- 
ing to the present invention can present an amage com- 
bining keyframes or key regions representing the entire 
contents of the scene without selecting a key frame rep- 
resenting a scene. Therefore, the synthetic key frame 
Fsk can summarize the entire contents of the scene. 
[0050] The synthetic key frame of the present inven- 
tion can be generated using the key frame or key region 
for entertainment, documentary, talk show, education, 
advertisement and home shopping programs as well as 
the news and sports video described above with refer- 
ence to FIGS. 11, 12A, 12B, 13A and 13B. 
10051] Meantime, if arrangement information of con- 
stituent elements of the synthetic key frame, such as key 
regions or key frames, Is described in the description, a 
user is able to not only browse corresponding video us- 
ing the synthetic key frame but al30 perform non-linear 
video browsing using the constituent elements. Since 
the synthetic key frame shown in FIG. 11 » for example, 
generated by combining the key regions Reg1 , Reg2, 
Reg3, Reg4 and Reg5 of the key frames extracted from 
the headline news section, the user selects a key region 
(Reg! , for instance) of the synthetic key frame so that 
he/she can browse a headline news item or detailed 
news item corresponding to the selected key region 
[0052] FIG. l4shows structural information of a video 
stream and a synthetic key frame that hierarchically 
summarizes the structural information in accordance 
with the present invention. In FIG 14 nodes correspond 
to frames representative of a program, shot and scene 
Nodes Na, Nb, Nc and Nd that are synthetic key frames 
that represent contents of lower level. To summarize 
lower structures, key regions or key frames of the lower 
level can be used for the synthetic key frames of upper 
structures. Accordingly, the user can searcWbrowse a 
video stream using a hierarchical structure of video at a 
desired level and the synthetic key frames. If one key 
frame or key region Is sleeted for nodes Na, Nb, Ncand 
Nd, a user can not fully understand the lower structure 
and content without browsing the lower level. But with 
synthetic key frame, user can easily understand the 
structure and content of the lower level without esxplicit 
browsing of the lower level. 

[0053] Hierarchical image summary elements must 
be defined in orderto summarize the video stream with 
the hierarchical structure. FIG. 15 shows the description 
structure of the hierarchical image summary element for 
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hierarchical video stream summary according to the 
present invention. The description structure Of the hier- 
archical image summary element, which is a recursive 
structure, includes variables such as a key image loca- 
5 tor, a list of sub-hierarchical image summary elements, 
summary level information and fidelity indicating how 
faithfully corresponding synthetic keyframe represents 
the lower structures Here, the key (mage locator is a da- 
ta structure capable of designating a keyframe, key re- 
10 gion and synthetic key frame, and the list of sub-hierar- 
chical image summary elements describes a lower sum^ 
mary structure, each element Df the list being a hierar- 
chical image summary element For example, When the 
number of the elements of the list of sub-hierarchical 
is image summary elements is '0\ it corresponds to the 
lowest node(leaf node) and means there does not exist 
a lower summary element any more 
[0054] FIG. 1 6 shows a non-linear video browsing in- 
terface exampf© using the synthetic keyframe accord- 
20 ing to the present Invention The video browsing inter- 
face Includes a video display view V-VD, a key frame/ 
key region view V-Fk/Reg, and a synthetic key frame 
view V-Fsk. The video display view V-VD and the key 
frame/key region view V-Fk/Reg are the same functions 
23 as those of the general non-linear video browsing inter- 
face shown in FIG- 3. Trie synthetic key frame view V* 
Fsk displays a video summary on a screen using the 
synthetic key frame such that the user can select the 
synthetic key frame or the key frame or key region in- 
50 eluded in the synthetic key frame to easily move to the 
section corresponding to the key frame or key region. 
The synthetic key frame view V-Fsk may be displayed 
one-dimensionalry, as shown in FIG- 16, or displayed in 
a TOC-shaped tree structure. 
35 p)055] Meanwhile, the synthetic key frame according 
to the present invention can be applied to UMA applies- ■ 
tion- Here, the UMA is an apparatus having Improved 
information transmission performance, which can proc- 
ess any of multimedia information into a form most suit- 
.40 able for a user environment, being adapted to a variety 
of variations in the user environment, to allow a user to 
be ableto conveniently use the inf ormation. Specifically, 
the user can obtain only limited information based on 
his/her terminal or a network environment connecting 
45 the terminal to a server. For instance, the device the user 
uses may not support motion pictures but still images, 
or it may not support video but audio, in addition, on the 
basis of network connection method/medium, there Is a 
limit in the amount of data capable of being transmitted 
w to the users device within a predetermined period of 
time because of insufficiency in the transmission capac- 
ity of data delivered through the network The UMA con- 
verts and transmits a video stream to a user who cannot 
receive and display the video stream due- to restriction 
ss conditions of the device/network, using reduced num- 
bers of key frames wUh a decreased size within the user 
environment. By doing so. the UMA can help the user 
to understand contents included in the video stream. 
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[0056] By being applied to the UMA, the synthetic key 
frame of the invention can be used as a moans for pro- 
viding a lot of meaningful information while reducing the 
number of the keyframes to be transmitted to decrease 
the amount of data to be delivered. 
[0057] FIG. 17 shows an example of application of the 
synthetic key frame according to the present invention 
to the UMA. This application includes a server £ gener- 
ating the synthetic key frame according to the present 
Invention, and a terminal T for receiving the synthetic 
key frame from the server S and transmitting a prede- 
termined request signal to the server. As described 
above, the synthetic keyframe Fsk consists of texts, key 
regions and key frames. 

[0058] FIG 1 8 is a flow diagram showing a method of 
receiving information using the synthetic key frame ac- 
cording to the present invention, which is applied to 
UMA Referring to FIG. 1 B, when the synthetic keyframe 
Fsk is sent from the server S to the user's terminal T, 
the userselects the synthetic keyframe or a component 
thereof, corresponding to a part he/she wants to browse, 
and then requests the server to deliver audio of corre- 
sponding part (ST1 )- When the server S sends the audio 
to the user, the user receives the audio and. when it Is 
not the information he/she wants, does not browse the 
contents Included in the synthetic key frame any more. 
However, if he/she wants to more information, he/she 
requests more keyframes with respectto corresponding 
section (ST2). By doing so, the user can browse the con- 
tents of the synthetic key frame more and he/she can 
also request the video to browse video streams (ST3). 
[0059] In case where the synthetic key frame is ap- 
plied to the UMA, the user can select a desired part and 
easily browse it so that he can save communication 
cost Furthermore, the server can easily transmit infor- 
mation aboutthe contents of multimedia stream to even 
a device with a limited function. 
[0060) As described above, the synthetic key frame 
of the present invention is generated by combining key 
frames or key regions to represent a specific section or 
segment of a video stream, thereby displaying lots of 
information on limited device. Moreover, the synthetic 
key frame can summarize a video stream one-dlmen- 
sionalry or hierarchically and it can be used as a means 
for non-Hnear video browsing. In addition, the synthetic 
key frame of the invention can be effectively applied to 
UMA with a limited performance of a terminal or trans- 
mitting device, and it can be also applied to all of the 
video genres. The video summarizing method using the 
synthetic key frame of the invention can efficiently sum- 
marize the content of avideo because it can sufficiently 
display the content of shots or scenes on a screen with 
a fixed size using the synthetic key frame 
[0061] Although specific embodiments including the 
preferred embodiment have been illustrated and de- 
scribed, It Wli be obvious to those skilled in the art that 
various modifications may be made without departing 
from the scope of the present invention. 



Claims 

1. A method of generating a synthetic key frame, com- 
prising the steps of: 

5 

receiving a video stream from afirst source and 
dividing it Into meaningful sections; 
selecting key frame(s) or key region(s) repre- 
sentative of a divided section; and 
io combining the selected key frame(s) or key re- 

gions), to generate one synthetic keyframe 

2. The method of generating a synthetic key frame as 
claimed in claim wherein the dividing step further 

15 comprises the step of receiving avideo stream from 
asecond source and dividing It Into meaningful sec- 
tions. 

3. The method of generating a synthetic key frame as 
20 claimed in claim 1 . wherein the selecting step fur- 
ther comprises the step of selecting key frame(s) or 
key region(s) output from the second source. 

4w The method of generating a synthetic key frame as 
25 claimed in claim 1 , wherein the section is a unit of 
segment 

5. A method of describing synthetic key frame data, 
comprising the steps of, 

30 

dividing a video stream into meaningful sec- 
tions, and synthesizing a keyframe or key re- 
gion representing the content of each section 
into one image, to generate a synthetic key 
35 frame; and 

describing a list of key frame and/or key region 
Included in constituent elements of the synthet- 
ic key frame. 

to 6. A method of describing synthetic keyframe data as 
claimed in claim 5, wherein the describing step in- 
cludes: 

an ID for identifying the synthetic key frame; 

45 a representative segment locator which de- 

scribe the temporal information of the segment 
that the synthetic key frame represent; and 
, key frame list or key region list for identifying 
the elements of the synthetic key frame; 

so wherein the describing Step can additionally in- 

clude 

a fidelity value indicating how faithfully the syn- 
thetic key frame represent the segment, and 
information on the arrangement of each constit- 
55 uent element when the key frame or key region 

is displayed as the constituent element of the 
synthetic key frame. 
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7. A method of describing synthetic key frame data as 
Claimed in claim 6, wherein the information about 
the arrangement Includes two-dimensional location 
information of the constituent element or layer in- 
formation as three-dimensional location information 
of the constituent element. 

8. A method of describing synthetic key frame data as 
claimed in claim 5. wherein, When the synthetic key 
frame includes the key frame fist, each element of 
the key frame list has a key frame locator as a key 
frame description unit structure and, when the syn- 
thetic key frame includes the key region list, each 
element of the key region list has a key region loca- 

" tor as a key region description unit structure 

9. A method of describing synthetic key frame data as 
claimed in claim 8, wherein the key frame locator 
includes an image locator capable of containing the 
location, annotation and a related segment with re- 
spect to a stored image, as data for designating the 
key frame, a segment locatorfor indicating informa- 
tion including a segment locator that designates a 
segment represented by corresponding key frame, 
and additionally a fidelity value indicating howfaith- 
f ully the key frame represents the segment. 

10. A method of describing synthetic key frame data as 
dairned in clafm 8; wherein the key region locator, 
serving as a data structure for describing the key 
region, is information logically* hysically designat- 
ing stored location or segment data, wherein the 
key region locator includes an inherent ID for iden- 
tifying the key region; 

an image locator and region area Info to locate 
the region or region data to locate the region, 
and 

a representative segment locator; 
wherein the key region locator can additionally 
include a fidelity value indicating how faithfully 
the key region represents the segment; 
an annotation, and 

a list of related segment with the key region. 

11 . A method of describing synthetic key frame data as 
claimed in claim 5, wherein, when the synthetic key 
frame Includes the key frame list, each component 
of the key frame I ist has fidelity indicating how f aith- 
fulfy corresponding keyframe represents the mean- 
ingful content In the synthetic key frame, as a key 
frame description unit structure, and, When the syn- 
thetic key frame includes the key region list, each 
component of the key region list has a fidelity value 
Indicating how faithfully corresponding key region 
represents the meaningful content in the synthetic 
key frame, as a key region description unit struc- 
ture. 



12. A method of describing synthetic key frame data, 
comprising the steps of 

dividfng a video stream Into meaningful sec- 
5 tions, and synthesfeing a key frame or key re- 

gion representing the content of each section 
into one image, to generate a synthetic key 
frame, and 

generating a combination of key frames or key 
10 regions, or key frame and key region included 

in constituent elements of the synthetic key 
■ frame, and physically Storing the combination 
to describe the synthetic keyframe. 

is 13. a method of describing synthetic keyframe data as 
claimed in claim 1 2, wherein the synthetic keyframe 
description includes. 

an ID for Identifying the synthetic key frame; 
20 animaga locatorfordesignatingthestoredsyn- 

thetlc key frame file, 

an ID for identifying the synthetic keyframe; 
an representative segment locator which de- 
scribe the temporal Information of the segment 
25 that the synthetic key frame represent; and 

key region list for identifying the elements of the 
synthetic key frame; wherein the description 
can additionally include 
a fidelity value indicating how faithfully the syn- 
50 thetic key frame includes section information 

about a segment represented by the synthetic 
key frame 

and information on the arrangement of the key 
frame and key region that are the constituent 
35 elements of the synthetic key frame. 

14. A method of describing synthetic key frame data as 
claimed in claim 12, wherein each element of the 
key region list of the synthetic key frame constituent 
40 elements has a key frame locator- or a key region 
locator. 



45 



SO 



55 



1S. A method of describing synthetic keyframe data as 
claimed in claim 14. wherein the key region locator, 
serving as a data structure for describing the key 
region, is information logically/physically designat- 
ing stored location or segment data, the key region 
locator includes; 

an inherent ID for identifying the key region, an 
Image locator and region area info to locate the 
. region or region data to locate the region; and 
a representative segment locator; 
wherein the key region locator can additionally 
Include 

a fidelity value indicating how faithfully the 
key region represents the segment; 
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an annotation; and 

a list of related segment with the key region 

1 6. A method of describing synthetic key frame data as 
claimed in claim 13, wherein each element of the 
key region list includes a fidelity value indicating 
how faithfully corresponding key region represents 
the meaningful content in the synthetic keyframe, 
as a key region description an ft structure. 

17- A method of describing synthetic key frame data as 
claimed in claim 13, wherein the Information about 
the arrangement includes two-dimensional location 
Information of the constituent elements or layer in- 
formation that is three-dimensional location infor- 
mation of the constituent elements 

1&. A hierarchical video summarizing method using a 
synthetic keyframe, comprising the steps of: 

dividing a video stream Into meaningful sec- 
tions, and synthesizing a key frame or key re- 
gion representing the content of each section 
into one image, to generate a synthetic key 
frame; and 

assigning the synthetic key frame to a key Im- 
age locator, a hierarchical summary list for de- 
scribing lower summary structures, and struc- 
tural information of the video stream. 

19. The hierarchical video summarizing method ustag 
a synthetic key frame as claimed in claim 1 3, where- 
in the key image locator is a data structure for des- 
ignating an image using a key region locator, a key 
frame locator and a synthetic key frame locator. 

20. The hierarchical video summarizing method using 
a synthetic key frame as claimed in claim 1 8, where- 
in each hierarchical summary structure is repre- 
sented by an image representative of a specific seg- 
ment 

21. The hierarchical video summarizing method using 
a synthetic key frame as claimed in claim 1 S, where- 
in each component of the lower hierarchical sum- 
mary list uses a hieratchccaVrecurslve summary 
structure as a lower hierarchical summary struc- 
ture. 

22. Trie hierarchical video summarizing method using 
a synthetic key frame as claimed in claim 1 8, where- 
in the hierarchical summery structure has summary 
level Information. 

23. The hierarchical video summarizing method using 
a sy iMhetie key frame as claimed in craim 1 8, where- 
in the hierarchical summary structure includes a fi- 
delity value indicating how faithfully a part, repre- 



2835A1 ™ 

rented by the lower hierarchical summary list is ex- 
pressed. 

24, A method for providing a video browsing interface, 
s comprising : 

dividing a video stream into meaningful sec- 
tions, and synthesizing a key frame or key re- 
gion representing the content of each section 
10 into one image, to generate a synthetic key 

frame; and 

providing a user interface to a predetermined 
display to browse a video related with the gen- 
erated synthetic key frame. 

15 

25. The method for providing a video browsing interface 
as claimed in claim 24, wherein the user interface 
provides the synthetic keyframe in the form Df view. 

20 ze. The method forproviding a video browsing interface 
as claimed in claim 24, wherein the synthetic key 
frame is arranged in a time sequence, and the syn- 
thetic key frame Is arranged In a tree shape. 

25 27. The methodf or providing a video browsing interface 
as claimed in claim 24, wherein the synthetic key 
frame is assigned to each node in TOO form 

28. A non-linear video browsing method, comprising 
so the steps of 

dividing a video stream into meaningful sec- 
tions, and synthesizing a key frame or key re- 
gion representing the content of each section 

39 into one image, to generate a synthetic key 
frame; 

providing a user interface to a predetermined 
display to browse a video related with the gen- 
erated synthetic key frame; 

40 selecting the synthetic key frame according to 
an input by a user; and 

reproducing a segment represented by the se- 
lected synthetic key frame 

45 29. The non-linear video browsing method as claimed 
in claim 28, wherein the reproducing step reproduc- 
es a segment related with constituent elements (key 
region or keyframe) of thecontents of the keyframe 
or the key frame selected by the user's Inpui- 

sa 

30. Apparatus comprising means adapted to carry out 
the method steps of any preceding claim. 

31. Video browsing apparatus comprising means 
as adapted to carry out the method steps of any of 

claims 1 to 29. 

32. A video browsing interface provided by carrying out 
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33. A computer program product comprising computer 
program elements adapted to carry out the method 
steps of any of claims 1 to 29. 5 
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